Method and apparatus for transmitting/receiving media contents in multimedia system

ABSTRACT

A method and an apparatus of transmitting media content in a multimedia system are provided. The method includes determining a transmission unit for transmitting at least one multimedia source related to the media content, generating a data stream including header information and the at least one multimedia source according to the determined transmission unit, and transmitting the data stream through a communication network.

PRIORITY

This application is a continuation application of prior application Ser.No. 13/495,429, filed on Jun. 13, 2012, which has issued as U.S. Pat.No. 10,110,655 on Oct. 23, 2018 and was based on and claimed priorityunder 35 U.S.C. § 119(a) of a Korean patent application number10-2011-0057714, filed on Jun. 14, 2011, in the Korean IntellectualProperty Office, a Korean patent application number 10-2011-0104862,filed on Oct. 13, 2011, and a Korean patent application number10-2011-0125458, filed on Nov. 28, 2011 in the Korean IntellectualProperty Office the disclosure of which is incorporated by referenceherein in its entirety.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present invention relates to a method and an apparatus fortransmitting and/or receiving media contents in a multimedia system.More particularly, the present invention relates to a method and anapparatus for transmitting and/or receiving media contents by usingvarious transmission units.

2. Description of the Related Art

Due to an increase of multimedia contents available through the Internetand communications networks, as well as recent developments oftechnology, hybrid media contents, including various types of multimediacontents, are released for viewing, listening, using, and consuming, ata time in which a single or two or more multimedia sources have beenused. The hybrid media contents may include various types of multimediacontents. The hybrid media contents are generated by separating thehybrid media contents into individual multimedia contents.

For example, the hybrid media contents may be implemented with anapplication, such as a widget for constructing an image that is a hybridmedia content including video data, image data, voice data, and aspecific motion image, and which are generated through an image signalencoding method, a voice signal encoding method, a file compressionmethod, or other similar data generation methods, in accordance with acharacteristic of each media, and the respective multimedia contents arereconstructed as the hybrid media contents at a reception side.

As described above, the conventional hybrid media contents are separatedinto a plurality of multimedia contents at a transmission side andprovided to the reception side in the separated form. The reception sidegenerates the hybrid media contents by reconstructing the plurality ofseparated multimedia contents. The use of the hybrid media contents isprovided through the separation into and the reconstruction of theplurality of the separated multimedia contents. However, due to anincrease of the use of hybrid media contents, a method of moreefficiently generating and transmitting and/or receiving the hybridmedia contents is needed.

Therefore, a need exists for a system and method for performing a selfdiagnosis of a device without the inconvenience caused when manuallyselecting a self diagnosis item from a computer or a user interface.

SUMMARY OF THE INVENTION

Aspects of the present invention are to address at least theabove-mentioned problems and/or disadvantages and to provide at leastthe advantages described below. Accordingly, an aspect of the presentinvention is to provide a method and an apparatus for efficientlytransmitting and/or receiving a media content by using varioustransmission units.

Another aspect of the present invention is to provide a method and anapparatus for transmitting and/or receiving a hybrid media content,including various multimedia data, by using a transmission unitappropriate to corresponding multimedia data.

In accordance with an aspect of the present invention, a method oftransmitting a media content in a multimedia system is provided. Themethod includes determining a transmission unit for transmitting atleast one multimedia source related to the media content, generating adata stream including header information and the at least one multimediasource according to the determined transmission unit, and transmittingthe generated data stream through a communication network.

In accordance with another aspect of the present invention, an apparatusfor transmitting a media content in a multimedia system is provided. Theapparatus includes a transmission unit for transmitting a data streamthrough a communication network, and a controller for determining atransmission unit for transmitting at least one multimedia sourcerelated to the media content, for generating a data stream includingheader information and the at least one multimedia source according tothe determined transmission unit, and for controlling transmission ofthe generated data stream.

In accordance with another aspect of the present invention, a method ofreceiving a media content in a multimedia system is provided. The methodincludes receiving a data stream of the media content through acommunication network, and determining a transmission unit of the mediacontent, identifying header information according to the determinedtransmission unit, and decoding the media content including at least onemultimedia source according to the identified header information.

In accordance with another aspect of the present invention, an apparatusfor receiving a media content in a multimedia system is provided. Themethod includes a receiving unit for receiving a data stream of themedia content through a communication network, and a controller fordetermining a transmission unit of the media content, for identifyingheader information according to the determined transmission unit, andfor decoding the media content including at least one multimedia sourceaccording to the identified header information.

Other aspects, advantages, and salient features of the invention willbecome apparent to those skilled in the art from the following detaileddescription, which, taken in conjunction with the annexed drawings,discloses exemplary embodiments of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certainexemplary embodiments of the present invention will be more apparentfrom the following description taken in conjunction with theaccompanying drawings, in which:

FIG. 1 is a diagram illustrating a layer structure for processing hybridmedia contents in an Motion Picture Experts Group (MPEG) Media Transport(MMT) according to an exemplary embodiment of the present invention.

FIG. 2 is a diagram illustrating a construction of a header of aMedia-Unit (M-Unit) according to an exemplary embodiment of the presentinvention.

FIG. 3 is a diagram illustrating a construction of a header of an assetaccording to an exemplary embodiment of the present invention.

FIG. 4 is a block diagram illustrating a construction of a transmissionapparatus for generating and transmitting media contents according to anexemplary embodiment of the present invention.

FIG. 5 is a block diagram illustrating a construction of a receptionapparatus for receiving media contents according to an exemplaryembodiment of the present invention.

FIG. 6 is a diagram illustrating a video frame configured with aplurality of slices according to an exemplary embodiment of the presentinvention.

FIG. 7 is a block diagram illustrating a structure of a system fortransmitting media contents by using various transmission unitsaccording to an exemplary embodiment of the present invention.

FIG. 8 is a diagram illustrating a construction of a Media Fragment Unit(MFU) according to an exemplary embodiment of the present invention.

FIG. 9 is a diagram illustrating a construction of an M-Unit accordingto an exemplary embodiment of the present invention.

Throughout the drawings, it should be noted that like reference numbersare used to depict the same or similar elements, features, andstructures.

DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENT

The following description with reference to the accompanying drawings isprovided to assist in a comprehensive understanding of exemplaryembodiments of the invention as defined by the claims and theirequivalents. It includes various specific details to assist in thatunderstanding but these are to be regarded as merely exemplary.Accordingly, those of ordinary skill in art will recognize that variouschanges and modifications of the embodiments described herein can bemade without departing from the scope and spirit of the invention. Inaddition, descriptions of well-known functions and constructions may beomitted for clarity and conciseness.

The terms and words used in the following description and claims are notlimited to the bibliographical meanings, but, are merely used by theinventor to enable a clear and consistent understanding of theinvention. Accordingly, it should be apparent to those skilled in theart that the following description of exemplary embodiments of thepresent invention is provided for illustration purpose only and not forthe purpose of limiting the invention as defined by the appended claimsand their equivalents.

It is to be understood that the singular forms “a,” “an,” and “the”include plural referents unless the context clearly dictates otherwise.Thus, for example, reference to “a component surface” includes referenceto one or more of such surfaces.

Hereinafter, the exemplary embodiments of the present invention will bedescribed based on Motion Picture Experts Group (MPEG) Media Transport(MMT) technology among technologies to which the exemplary embodimentsof the present invention may be applied. However, the present inventionis not limited thereto, and the exemplary embodiments of the presentinvention may be applied to any suitable or similar technology orcommunications standard.

FIG. 1 is a diagram illustrating a layer structure for processing hybridmedia contents in the MMT according to an exemplary embodiment of thepresent invention.

Referring to FIG. 1, a structure of an E layer is shown, and the E layerincludes a MMT E.3 layer 105, MMT E.2 layer 107, and MMT E.1 layer 109.

Media Codec A 101 and media code B 103 are media codec layers fordecoding and/or encoding multimedia data. The media codec A 101 and themedia codec B 103 may have different functions. For example, a NetworkAdaptation Layer (NAL) unit for providing characteristic information,signaling information for decoding, and other similar information onmultimedia data, together with the multimedia data, including compresseddigital video, as control information, is an output of an encoder in aH.264 codec (not shown), and multimedia data, such as compressed digitalvideo, is an output of an encoder in the H.262 codec (not shown). H.264encoding may be performed in the media codec B 103, and an H.262encoding may be performed in the media codec A 101. However the presentinvention is not limited thereto, and the H.264 encoding may beperformed in the media codec A 101.

That is, the media codec C 103 is able to perform the encoding of themultimedia data to which control information, such as characteristicinformation and signaling information, is added, and the media codec A101 is able to perform the encoding of only the multimedia data.

Referring to FIG. 1, the MMT E.3 layer 105 adds control information,such as characteristic information and signaling information requiredfor the decoding, corresponding to multimedia data to the encodedmultimedia data transferred from the media codec A 101 and transfers themultimedia data to the MMT E.2 layer 107. In such a case, information onthe MMT E.3 layer 105 contains an NAL unit header and NAL unit data, andadditionally includes additional header information.

FIG. 6 illustrates a video frame configured with a plurality of videoslices according to an exemplary embodiment of the present invention.

Referring to FIG. 6, a structure of the video frame is for one framedisplayed as a single image of a video at a specific time and includes aplurality of slices 601 to 613. In further detail, the single image ofthe video is encoded into seven small blocks, which are the slices 601,603, 605, 607, 609, 611, and 613.

That is, the plurality of slices 601 through 613 are included in theframe displayed in the single image of the video. The plurality ofslices 601 through 613 may be arranged sequentially from a left sidetowards a right direction, as illustrated in FIG. 6, and may be arrangedfrom a top side towards a lower direction while filling the frame, ormay be arranged in any other suitable manner. For example, a pluralityof slices included in the single frame may fill the frame with variouscombinations in a direction from a left or right side towards a right orleft side or a direction from a lower or upper side towards an upper orlower side. The plurality of slices 601 through 613 may have differentlengths.

The frame having the structure of FIG. 6 may be configured as a MediaUnit (M-Unit) which is one of transmission units of the hybrid mediacontents, including at least one piece of information, such as controlinformation, among time information, an importance degree, and variousfunction information, such as a random access point to be described.Additionally, each of the slices 601 through 613 included in the frameis independently decodable and may be configured as a Media FragmentUnit (MFU).

Furthermore, the M-Unit may be defined as an independently operatingunit, such as a largest group frame, or in other words, a Group ofPictures (GoP), in which data has subordinate data units to be recorded.According to the present exemplary embodiment, when the MFU correspondsto one slice within the frame, the one frame is configured as oneM-Unit. Furthermore, when the MFU corresponds to one frame, the M-Unitis configured as the GoP that is an independent decoding unit.

In consideration of a next-generation multimedia broadcasting system, avideo frame may include the MFU that is a minimum decoding unit and theM-Unit that is the decoding unit including the control information.Furthermore, the M-Unit may include a single MFU, fragmented MFUs, or agroup of MFUs. The decoder may control a control function for eachminimum construction unit based on the control information. Also, theMFU is a minimum data unit input to the MMT E.3 layer 105 (see FIG. 1),and the M-Unit is an output data unit of the MMT E.3 layer 105 includingthe control information, such as the time information. Accordingly, thepresent exemplary embodiment allows for configuration of efficient mediacontents data for transmission and storage in a next-generationmultimedia broadcasting system by using the structuralized MFU orM-Unit.

The MFU according to the present exemplary embodiment is a minimumconfiguration unit for configuring the media contents data and includesconstruction information on the media contents data. The configurationinformation includes at least one of an identifier indicating a startpoint, a center point, and an end point of the respective media contentsdata, and indicating non-partition of data, setting information forsetting a media codec, type information indicating a type of datarepresenting information on a format of encoded media, priorityinformation on the media contents data, point information indicating astart point of the media contents data, size information on the mediacontent data, and information indicating specific information on thelayered encoding and a multi-view media codec. A unit size of the MFUmay indicate a size of an independently encodable and/or decodable unit.For example, a video may be fragmented into a picture, a slice, and anaccess unit, based on a data unit from a macro block that is a minimumencoding and/or decoding unit of a media codec.

FIG. 8 is a diagram illustrating a construction of an MFU according toan exemplary embodiment of the present invention.

Referring to FIG. 8, the present invention is not limited to theexemplary embodiment of FIG. 8 and the illustrated construction of theMFU, and the MFU may be configured in a variety of suitableconstructions and types.

The additional header information as the configuration information onthe MFU that is unit information included in the output data unit of themedia encoding includes at least one of the information fields 801 to809 in relation to the MFU. The information fields 801 to 809 aredescribed in further detail below.

As indicator 801 of the MFU includes the configuration information onthe MFU and indicates flag information indicating whether MFU data isfragmented and also indicates a start unit, a continued unit, or a lastunit among the MFUs. A data type 802 indicates a type of MFU data and isinformation indicating an attribute of corresponding MFU data. Forexample, the data type 802 may indicate the type of data for settinginformation on a media codec, motion prediction information of theencoding, encoded data, configuration information on encoded data, andother similar data.

A decoding order number 803 is used to indicate when flag information onthe fragmented MFU is continued. The MFU has an increasing number thatis order information indicating an order of the use of a correspondingMFU. Furthermore, a client terminal may utilize the increasing numberthat is the order information indicating the order of the use of thecorresponding MFU as information on a decoding order. Byte data offset804 indicates an actual position that is a start of media data. Length805 is information indicating a length of media data.

Priority (P) 806 is information that indicates a value of a priority ofthe MFU. For example, when the priority 806 is “0”, it indicates thatthe corresponding MFU does not have priority data, and when the priority806 is “1”, the corresponding MFU has priority data. In a case of an NALheader parameter for media, the priority 806 may be set as “1”. Thepriority 806 indicates that data fragmented within a single media unitis priority data, such that when the priority information is lost, thedecoding may be interrupted.

Dependency counter 807 indicates a value of a dependent MFU count withinan independently encodable and/or decodable unit. For example, when avalue of the dependency count is “4”, it means that four following MFUsare dependent on corresponding MFU data. For example, when there arefour correlated MFUs of the MFU having a dependency count of “4”, thefirst MFU has a count value of “3”, the second MFU has a count value of“2”, the third MFU has a count value of “1”, and the last MFU has acount value of “0”. Through the use of the dependency count, a degree ofpropagating an error may be indicated when the corresponding MFU datahas the error.

Multi-layer information 808 indicates an identifier of basic media, anidentifier of enhanced layer media in the layered video encoding, amulti-view media coding, and other similar information. The MFUs arerelated to information on the scalable media coding or the multi-layeredmedia coding. Media codec device additional information (Mi) 809 isinformation that is changed according to a device in an encoding processby using a corresponding media codec. This displays a list or extendedinformation on corresponding information in consideration of a casewhere a profile and level information on a corresponding media codec ischanged or a type of codec is changed.

Furthermore, the MMT E.3 layer 105 generates the M-Unit for addingcontrol information, such as characteristic information and signalinginformation required for the decoding, on corresponding multimedia datato the encoded multimedia data transferred from the media codec A 101and transferring the multimedia data to the MMT E.2 layer 107. Anexample of the control information included in the aforementioned M-Unitwill be discussed with reference to FIG. 9.

FIG. 9 is a diagram illustrating a construction of the M-Unit accordingto an exemplary embodiment of the present invention.

Referring to FIG. 9, an exemplary embodiment of the M-Unit, andinformation fields 901 to 908 are described below. However, the presentinvention is not limited thereto, and the M-Unit may be configured in avariety of suitable manners and types.

A type 901 is an M-Unit identifier indicating configuration informationfor the M-Unit. This represents flag information indicating whether theM-Unit includes a single MFU, fragmented MFUs, an MFU group, or aplurality of Access Units (AUs), as well as flag informationadditionally indicating whether the M-Unit is a start unit, a continuedunit, or a last unit. A sequence number 902 indicates when the flaginformation is continued. The M-Unit has an increasing number that isinformation on an order of its use.

A data offset 903, which is given in bytes, is information thatindicates a start position of M-Unit media data, and may be expressed ina length of header information for the M-Unit. Length information 904indicates a length of M-Unit media data. Timing information 905indicates a reproduction time of the M-Unit. The timing information 905on the reproduction time includes decoding time information, expressiontime information, and other similar timing information, and anexpression method may include a network synchronization time informationexpression method and a period included time information expressionmethod.

Random Access Point (RAP) 906 indicates random access point informationfor a corresponding M-Unit. Video data using the RAP 906 may provide arandom reproduction function. In the case of providing the RAP 906, theM-unit includes at least one piece of RAP 906 information. Furthermore,when the M-unit includes at least one piece of RAP 906 information, theRAP provides multiple pieces of location information corresponding tothe RAP. According to circumstances, a method of sequentially searchingfor a corresponding point by inserting a RAP recognition code of the MFUis provided.

Initial media codec information 907 indicates information on initialsetting values of a media codec. An initial setting for the decoding inthe media codec may be progressed through corresponding structureinformation.

Counter 908 may include information on allowance of multiple MFUs and aMFU/AU number. The multiple MFUs allowance information indicates whetherM-Unit data includes a plurality of MFUs or one MFU, and indicates thenumber of MFUs when the M-Unit data includes a plurality of MFUs. TheMFU/AU number indicates the number of internal data units when theM-Unit data includes a plurality of MFUs or AUs. Furthermore, the MFU/AUprovides corresponding position information on data for an access of acorresponding MFU or AU. Additionally, corresponding positioninformation may be found by adding an indicator for providing thecorresponding position information to header information on the MFU orthe AU and sequentially searching for the corresponding positioninformation.

As described above, the MMT E.3 layer 105 (see FIG. 1) adds the controlinformation to multimedia data, such as the MFU, acquired from a source,the MFU being transferred from the media codec A 101 and transfers themultimedia data to the MMT E.2 layer 107. Accordingly, the MMT E.2 layer107 constructs the M-Unit by aggregating at least one multimedia dataincluding the added control information.

The control information may be classified into control information aboutthe MFU and control information about the M-Unit. Additionally, aparticular information configuration of each of the control informationabout the MFU and the control information about the M-Unit may beconfigured in the manner described above, with respect to FIGS. 1, 6, 8and 9. Additionally, the control information may be added to the headerinformation in various forms. An exemplary embodiment of theconfiguration of the header of the M-Unit will be described below, withreference to FIG. 1.

The MMT E.2 layer 107 of FIG. 1 generates MMT asset data, hereinafter,referred to as an asset, by aggregating various types of MFUs providedfrom one or more multimedia data from multimedia sources andconstructing the aggregated MFUs into the M-Unit. The MMT asset isconstructed into hybrid media contents by receiving an input of a unitstructure in accordance with attributes of various types of multimediasources, i.e., multimedia data, such as a video data, audio data, textdata, file data, widget data, application data, and other similar data.The asset includes one or more MMT M-Unit data, each one of which is anM-Unit, that is a group of the aforementioned NAL units such that theMMT M-Unit data are discriminated from each other. The MMT E.2 layer 107generates the hybrid media contents in a form of the asset based oninformation necessary for transmission of the hybrid media contents orattribute information on multimedia data. Here, the “group” may beunderstood as a unit of encoded and/or decoded information in themultimedia system.

The M-Unit, which is a data group including information units having thesame characteristic information, is generated by adding additionalcharacteristic information required for the generation of the hybridmedia contents to the NAL Unit data that is the output of theaforementioned encoder. An example of the additional characteristicinformation includes lossless information that is a characteristic of afile or widget data and loss allowance and delay information that is acharacteristic of media data. The M-unit that is the group of the samecharacteristic units is configured and generated. The MMT E.1 layer 109of FIG. 1 configures one or more assets transferred from the MMT E.2layer 107 as an MMT package usable by a user.

FIG. 2 is a diagram illustrating a construction of a header of theM-Unit according to an exemplary embodiment of the present invention.

Referring to FIG. 2, fields included in the header of the M-Unit in FIG.2 will be described below.

M-Unit Identifier (ID) 201 is an identifier for determining a type ofM-Units. Sequence number 203 indicates an order according to a functionof the M-Unit. The sequence number 203 may be used for a function of anorder re-arrangement according to a media reproduction time, such as atransmission loss retransmission request or media fast forwardreproduction and a random access.

Type 205 indicates a type of data of the M-Unit. The type 205 mayindicate characteristic information on a data group, and a filecharacteristic considering a loss rate, delay information for mediareproduction, or other similar characteristics and information. Althoughit is not illustrated in FIG. 2, a payload onto which the data of theM-Unit is loaded is included at a backside of, or after, the header, sothat multimedia data having various sources, such as a video, an audio,a text, a file, and a widget, may be transmitted. Accordingly, the type205 may indicate a type of contents transmitted together with the headerof the M-Unit.

The type 205 may include a three digit field indicating the type suchthat 000 indicates H.264 Video, 001 indicates AAC Audio, 002 indicatestext, 003 indicates application, 004 indicates web pages, 005 indicatessignaling for codec, 006 indicates signaling for composition, and 007 isreserved for future use. The type of appropriate contents may beindicated by using a predetermined value within a determined range by aprovider of the hybrid media contents. Another example of the type 205may be a case where 000 indicates a H.264 I frame, 001 indicates a H.264B frame, and 002 indicates a H.264 P frame and may be used for a methodof assigning a priority to various sources of specific multimedia. AQuality of Service (QoS) may be considered in transmitting multimediadata based on the priority.

A Fragment Unit (FU) flag 207 indicates information on continuedM-Units. A value of the FU flag indicates whether a corresponding M-Unitis combined, is fragmented, or is a single M-Unit. A time flag 209indicates that a corresponding M-Unit includes time information. A GoP211 indicates that a corresponding M-unit includes information on itsGoP unit. When the GoP 211 is included, a value of the GoP 211 indicatesthat the corresponding M-Unit has a single AU or multiple AUs. AMFU-range, although not illustrated in FIG. 2, indicates a value of agroup of MFUs. A Spatial Parameter Set (SPS) 213 indicates that anM-Unit includes spatial parameter set information. The SPS 213 indicatesinformation on a spatial position in which data of a correspondingM-Unit is displayed. A Presentation Parameter Set (PPS) 215 indicatesthat an M-Unit includes PPS information. The PPS 215 is information usedfor codec initialization and contains specific information on a mediacodec. Furthermore, the SPS 213 and the PPS 215 use a general value ofan initialization parameter set of a media coding.

A sync 217 indicates that a corresponding M-Unit includessynchronization control information. The sync 217 is data for a mutualsynchronization between combination processed multiple M-Units and, forexample, is utilized as a synchronization marker between an M-Unit for aRight (R) image and an M-Unit for a Left (L) image for a 3-Dimensional(3D) image. Furthermore, sync 217 may be used to assign commandinformation between corresponding M-Units. The command information iscontrol information for the corresponding M-Units, and may indicate thata corresponding M-Unit corresponds to information such as addition,deletion, replacement, or update information. The Sync information maybe used as control information for only the M-Unit in the same sessiontransmitted through a hybrid transmission network based on the commandinformation.

A Transport Characteristic (TC) 219 indicates that an M-Unit includestransport characteristic information. The TC 219 is for transmitting acorresponding M-Unit and indicates, for example, a loss rate, a delaytime, parity information for an error recovery, an average bit rate fordata transmission, and a maximum bit rate. A RAP 221 indicates that acorresponding M-unit includes flag information for a random access. Afunction construction, such as a fast forward reproduction, may beprovided according to existence or non-existence of a correspondingflag.

FIG. 2 illustrates a construction of the header of the M-unit. However,the present invention is not limited thereto, and the construction ofthe header of the M-Unit according to the exemplary embodiment of FIG. 2may be of other suitable forms and constructions. For example, theconstruction of the header of the M-Unit may include a combination ofone or more information among the multiple header information includedin the header of the M-Unit described with reference to FIG. 2 and/orthe multiple control information added in generating the M-Unit in thedescription of the MMT E.3 layer 105 of FIG. 1.

FIG. 3 is a diagram illustrating a construction of a header of the assetaccording to an exemplary embodiment of the present invention.

Referring to FIG. 3, the asset, according to the present exemplaryembodiment, is generated in a structure including at least one M-Unitaccording to various multimedia data of media sources, and may beunderstood as a data stream including the various multimedia data ofmedia sources. That is, each multimedia source is generated as the MFUor the M-Unit and the asset includes at least one M-Unit.

Table 1 illustrates an example of the M-Unit, the asset, and the FU.

TABLE 1 Asset Header 

M unit headers 

 : M unit header 

 : M unit header 

 : M unit header 

 : 1) Asset type: A 

mfu rangue = 1-2 

mfu rangue = 3-4 

mfu rangue = 5-6 

mfu rangue = 5-6 

Asset Header 

Asset Header 

MPEG-2 TS data 

MP4 data 

2) Asset type: B 3) Asset type: C (MPEG-2 TS) 

 : 

(MP4) 

mfu header (1) 

 : mfu header (2) 

 : mfu header (3) 

 : mfu header (4) 

 : mfu header (5) 

 : off-set 

 : off-set 

 : off-set 

 : off-set 

 : off-set 

 : lengths 

length 

length 

length 

length 

mfu data (1) 

mfu data (2) 

mfu data (3) 

mfu data (4) 

mfu data (5) 

In Table 1, off-set means the byte data offset information.

In further detail, FIG. 3 illustrates an example of header informationon the asset used when the MMT E.2 layer 107 of FIG. 1 generates theasset with an input of the M-Unit that is grouped data of the sameattribute information of various types of multimedia data. Furthermore,the header of the asset of FIG. 3 may indicate a unit group of spatiallycontinued M-Units. An asset ID 301 indicates an identifier forcorresponding continuous data. The asset ID 301 is identificationinformation on corresponding assets in a process of generating thehybrid media contents.

Asset Length information 303, 305, 307, and 311 indicate a length ofdata of a corresponding asset. The asset length information 303, 305,307, and 311 may be set to have a sufficiently long length consideringUltra Definition (UD) video data. Asset Length information 303, 305,307, and 311 are separately indicated in three areas for convenience'ssake. However, the present invention is not limited thereto, and theAsset Length information 303, 305, 307, and 311 may be one field or anysuitable number of fields.

An asset type 309 indicates a data type of a corresponding asset, andmay use the data type of the M-Unit aforementioned in FIG. 1 in the samemanner as described with reference to FIG. 1. However, a multimedia typemay be indicated in various methods, in addition to the aforementionedmethod. For example, a type of a first multimedia source may be directlynotified to prevent inaccurate decoding of a multimedia source only witha name of a representative multimedia source due to recent variousprofiling and use of an audio codec, a video codec, or variousapplications. For example, there are approximately 10 types of profilingin H.264, so that when a type of asset is simply notified through H.264,an error may be generated in a decoding process. Furthermore, whenmultiple multimedia data is multiplexed and used in configuring the MMTpackage, a type of asset having a long length is used.

Additionally, for example, a type of asset having a length of 13 bitsmay conform to the data type of the M-Unit and it may indicateadditional information according to a corresponding priority by addingan extension flag to the asset according to circumstances. For example,in the case of “00000000” based on H.264, an I frame in an image ofH.264 may be indicated by “001”, a B frame thereof may be indicated by“010”, and a P frame thereof may be indicated by “100”. A transmissionpriority for guaranteeing a maximum QoS at the transmission side fortransmitting the M-Units may be considered based on the priority.

An M-Unit control information header length 313 indicates a length of anM-Unit control information header indicated in a corresponding payload,and notifies that there is M-Unit control information data equal to thelength of the M-Unit control information header. M-Unit controlinformation 315 includes at least one of synchronization information andasset command information, for commands to add, delete, insert, update,and replace, considering the hybrid transmission network, a time scalethat is a basic time unit of a corresponding asset data group, transportcharacteristic information indicating a transmission rate, a loss rate,a delay allowance time, a maximum bit rate, an average bit rate, whetherto guarantee a QoS, and other similar information, for the transmissiondata group, the number of M-Units included in a data group of acorresponding asset, and additional information on the RAP.

Although the header having a specific length, such as 1 bit or 2 bits,has been described as an example for describing a difference of therespective exemplary embodiments, a header of the M-Unit and/or assethaving other lengths may be used. Furthermore, when the fields, asdescribed above in the exemplary embodiments, are used without regard toa length, the configuration method of the M-Unit and/or asset may bedone according to the exemplary embodiments of the present inventionwithout regard to the length, as described in the exemplary embodiments.

The identifier or data type of the asset according to the exemplaryembodiments of the present invention may indicate a type of multimediadata of a media source included in the M-Unit of the asset. For example,the identifier or data type of the asset may indicate whether the assetis for video data, audio data, or that the asset includes multiplemultimedia sources, such as a video data, audio data, file data, widgetdata, or any other similar type of data. Furthermore, the identifier ordata type of the M-Unit may indicate video data, audio data, file data,a widget having a type of data, i.e., a multimedia source, included in acorresponding M-Unit.

FIG. 7 is a block diagram illustrating a structure of a system fortransmitting media contents by using various transmission unitsaccording to an exemplary embodiment of the present invention.

Referring to FIG. 7, a structure of the data stream may be classifiedinto a structure of a streaming payload 701 for transmitting real timedata closely related to a real-time broadcasting and a time and may alsobe classified into a file transfer payload 703 for transmitting non-realtime data, such as a file transmission. Furthermore, in the streamingpayload 701, media contents may be configured and transmitted using theMMT M-Unit or the MMT MFU as a basic transmission unit for real timedata transmission.

An MFU 705 is a basic unit for processing multimedia data input from acodec or an application source. The MFU 705 may be one of an image or aplurality of blocks included in one image in a video codec, and may beone file block in a file, such as an application. Furthermore, the MFU705 may be an MMT M-Unit 707 that is a basic unit of data transmissionor application driving. For example, one M-Unit 707 may be constructedwith a group of several AUs, which are the units for transmission ofdata configuring one image in video multimedia data. Through theconfiguration of the M-Unit 707, a user may perform a fast forwardsearch or search through a video.

Furthermore, the M-Units 707 may be configured as one MMT asset 709.Further, the MMT asset 709 may be one of independent video data, audiodata, or subtitles. A group of various types of MMT assets 709 may be anMMT package 711 capable of providing hybrid multimedia contents.Additionally, a data stream for file transmission is used fortransmitting non-real time data, and a transmission unit may be anentire or a specific part of the MMT asset 709 or the MMT package 711 ofthe hybrid media contents.

As described above, a unit for encoding the media contents in thestructure for transmitting the media contents according to the exemplaryembodiment of FIG. 7 may be the MFU 705 or the M-Unit 707. In a case ofan independently decodable video, the MFU 705 refers to one frame forthe encoding or one of a plurality of slices when the frame includes theplurality of slices. Furthermore, in a case of an independentlydecodable video, the M-Unit 707 refers to one frame for the encoding ora GoP that is an independently decodable unit for the encoding.

A transmission unit of media contents may be determined considering anattribute and a service of the media. The asset 709 is a stream of thecontinuous M-Units 707 configured in the same media type. The package711 is a form of a single asset or a group of multiple assets and mayinclude additional information, i.e., image configuration informationtime of hybrid media, transmission environment information considering aspatial arrangement order and a hybrid network environment, a demandedtransmission bandwidth, a loss probability, information on whether toallow a delay time, or any other suitable or similar information for aservice.

FIG. 4 is a block diagram illustrating a construction of a transmissionapparatus for generating and transmitting media contents according to anexemplary embodiment of the present invention.

Referring to FIG. 4, the transmission apparatus includes a multimediasource input unit 401, a multimedia source identification unit 403, amultimedia source storage unit 405, a transmission unit 407, and acontroller 409. The transmission apparatus of FIG. 4 supports varioustransmission units described with reference to FIG. 7, determines atransmission unit according to the structure for streaming fortransmitting real time data and the structure for file transmission fortransmitting non-real time data, and configures one or more multimediasources into the MFU, the M-Unit, the asset, or the package as thetransmission unit according to the determined transmission unit andtransmits them as such. The multimedia source input unit 401 receives aninput of the M-Unit that is a group of the various types of multimediadata as a multimedia source for generating the hybrid media contents.The multimedia source identification unit 403 assigns an asset ID to thegroup of the M-Units as the multimedia source, identifies a type, e.g.,a voice source, a video source, a file, or an execution file, of themultimedia source, and assigns a corresponding asset ID to the group ofthe M-Units. Here, one asset may include multiple M-Units including thevarious multimedia sources. The multimedia source storage unit 405stores the generated assets according to the configuration method of theasset. The transmission unit 407 transmits the asset or assets through acommunication network. The controller 409 configures the M-unit byaggregating various types of MFUs provided from one or more multimediadata of multimedia sources and performs a general control for generatingand transmitting the MFU, the M-unit, and the asset.

FIG. 5 is a block diagram illustrating a construction of a receptionapparatus for receiving media contents according to an exemplaryembodiment of the present invention.

Referring to FIG. 5, the reception apparatus includes a receiving unit501, a multimedia source reading unit 503, a multimedia source storageunit 505, a display unit 507, and a controller 509. The receptionapparatus of FIG. 5 supports the various transmission units describedwith reference to FIG. 7, determines a transmission unit of mediacontents received by the corresponding transmission units according tothe structure for the streaming for transmitting the real time data andthe structure for file transmission for transmitting the non-real timedata, identifies header information on the transmission unit, andperforms the decoding of such. The receiving unit 501 of FIG. 5 receivesthe asset including one or more M-Units, including one or more MFUs,according to various multimedia data of multimedia sources through acommunication network, and the header information on the asset and theheader information on the M-Unit is transferred to the controller 509 tobe used for the reception control and the decoding of the hybrid mediacontents. Furthermore, the controller 509 performs a general controlrelated to the reception and the decoding of the hybrid media contentsby using the control information about the MFU and the controlinformation about the M-Unit.

The multimedia source reading unit 503 reads a data type of eachmultimedia source in the unit of the M-Unit based on the controlinformation, including the control information about the MFU,transferred from the controller 509 receiving the respective informationabout the asset and the M-Unit, and classifies and stores the respectiveread multimedia sources in the source storage unit 505. The controller509 of FIG. 5 performs a general control for the reception of the assetand the reading and the decoding of the multimedia source, and data ofthe decoded multimedia source is output through the display unit 507.

Accordingly, the present exemplary embodiments described above mayprovide the apparatus and the method for transmitting and/or receivingmedia contents by using various transmission units, such as the MFU, theM-Unit, the asset, or the package. Furthermore, the present exemplaryembodiments may provide the apparatus and the method for efficientlygenerating and transmitting and/or receiving the hybrid media contentsincluding various types of multimedia sources, i.e., multimedia data.Furthermore, the present exemplary embodiments may appropriatelyfragment multimedia data having a predetermined size in transmitting orstoring the multimedia data.

Furthermore, the present exemplary embodiments may easily identify aposition or a transmission time point of the M-Unit that is the decodingunit including at least one among a time, an importance degree, andvarious function information, as the control information, in themultimedia system and may efficiently notify a format of the asset,which is one format of the hybrid media contents, using an appropriatemanner. Furthermore, the present exemplary embodiments may efficientlynotify the type of asset in the multimedia system.

While the invention has been shown and described with reference tocertain exemplary embodiments thereof, it will be understood by thoseskilled in the art that various changes in form and details may be madetherein without departing from the spirit and scope of the invention asdefined by the appended claims and their equivalents.

What is claimed is:
 1. An apparatus for receiving a media packet in amultimedia system, the apparatus comprising: a receiver configured toreceive data; and a processor coupled with the receiver and configuredto: receive the media packet including data of a unit including at leasta part of media data in a payload of the media packet, the unitincluding at least one media fragment unit (MFU) which is a fragment ofthe unit, and decode the media packet including the data of the unit,the unit being independently decodable in decoding processing, whereinthe MFU includes a count value indicating a number of at least onesubsequent MFU which is dependently decodable based on the MFU in thedecoding processing, the at least one subsequent MFU being correlatedwith the MFU in the decoding processing, wherein the MFU includes numberinformation indicating an order of the MFU, wherein the MFU includesinformation indicating a priority of the MFU relative to another MFUwithin the unit, and wherein the payload includes an indicatorindicating one of: the payload includes one or more complete MFUs, thepayload includes a first fragment of the MFU, the payload includes alast fragment of the MFU, or the payload includes a fragment of the MFUthat is neither the first fragment nor the last fragment.
 2. Theapparatus of claim 1, wherein the count value indicating the number ofthe at least one subsequent MFU is included in a header of the MFU. 3.The apparatus of claim 1, wherein the media data comprises timed data.4. The apparatus of claim 1, wherein the at least one subsequent MFU isnot processed without the MFU.
 5. The apparatus of claim 1, wherein thecount value of the MFU does not indicate a decoding order of the MFU.